Since the exposition of several input frames (snapshots), as a rule, occurs at different moments in time, there are distortions in forming a scene between various frames that is manifested in the different disposition of moving (unsteady) objects and in the change of conditions of scene lighting (for instance, as a result of changes in cloudiness), which affects the quality of a resulting image and is manifested in:
Double contours of unsteady objects and, in some cases, in doubling the number of moving objects in an image
Semi-transparency of unsteady objects
Noticeable irregularity in brightness or color balance of various areas of the image
Currently, one of the ways for improving the quality of pictures with the help of mobile devices is to shoot several frames in a short period of time, subsequently merging them by stitching several images into a single resulting photograph (creating a panoramic image, an image with enhanced dynamic range, decreasing the level of image noise, etc.). In doing this, stitching should be done in such a way that seams pass through the trajectory of the least differences in neighboring initial images and that they bypass unsteady objects. A widespread method for determining an optimum seam trajectory uses various image segmentation methods [Alexander Vezhnevets, Olga Barinova, “Image Segmentation Methods: Automatic Segmentation”, Computer Graphics and Multimedia, Issue No. 4(4)/2006: http://cgm.computergraphics.ru/content/view/147].
Image segmentation and stitching is most suitable in the following cases of creating one image out of several initial images:
Creating a panoramic image out of several shots, each of which depicts only part of a panorama
Creating an image with a high dynamic range out of several initial low dynamic range pictures
There are various methods for improving the quality of images with the use of segmentation.
The clustering method uses a representation of image points that is assigned to a certain feature space, and this feature space introduces metrics (measure of closeness).
The disadvantage of this method consists in the fact that the spatial arrangement of points is either not taken into account at all or is taken indirectly (for instance, using the point's coordinates as one of the features). That is why, usually after clustering the image points, a procedure for defining connected components is performed. Moreover, clustering methods do not work well on noisy images: often separate region points are lost, several small regions are created, and so on.
The method of region growing [A. Tremeau and N. Borel, “A Region Growing and Merging Algorithm to Color Segmentation”, Pattern Recognition, 1997; Y. Kanai, “Image Segmentation Using Intensity and Color Information”, SPIE—Visual Communications and Image Processing '98; B. Cramariuc, M. Gabbouj, and J. Astola, “Clustering Based Region Growing Algorithm for Color Image Segmentation”, International Conference on Digital Signal Processing, 1997; and Y. Deng, B. S. Manjunath, and H. Shin, “Color Image Segmentation”, CVPR 1999] takes into account the spatial arrangement of points directly. First, according to a certain rule, the regions' centers are selected, and adjacent points which meet a certain criterion are added to them in stages. The region growing process stops when none of the image points can be added to any of the regions. There are different criteria that are used, on the basis of which a point is added or not added to a region: closeness of a point to a region center, closeness to an adjacent point added to a region at the previous step, closeness according to certain region statistics, cost of the shortest path from a point to a region center, etc. For the most part, the region growing process is utilized to cover separate regions. However, following this procedure step by step or simultaneously for several regions, it is possible to obtain partitioning of a whole image.
The disadvantage of this method is its inapplicability to tasks of seam stitching; it is applied only in cases where there is only one reference image. Also, it requires significant memory resources of mobile devices. Moreover, the speed of the data processing is not fast enough.
Fragmentation-merging methods [A. Tremeau and N. Borel, “A Region Growing and Merging Algorithm to Color Segmentation”, Pattern Recognition, 1997; B. Cramariuc, M. Gabbouj, and J. Astola, “Clustering Based Region Growing Algorithm for Color Image Segmentation”, International Conference on Digital signal Processing, 1997; M. Celenk, “Hierarchical Color Clustering for Segmentation of Textured Images”, Proceedings of the 29th Southeastern Symposium on System Theory, 1997]; [S. Ji and H. W. Park, “Image Segmentation of Color Image Based on Region Coherency”, Proceedings of ICIP '98; L. Shafarenko, M. Petrov, and J. Kittler, “Automatic Watershed Segmentation of Randomly Textured Color Images”, IEEE Transactions on Image Processing, 1997; and M. Barni, S. Rossi, and A. Mecocci, “A Fuzzy Expert System for Low Level Image Segmentation”, EUSIPCO-96] consist of two main stages: fragmentation and merging. Fragmentation starts with a partitioning a certain image but not necessarily into uniform regions. The region fragmentation process is carried out until there is a partitioned image which meets the requirement of segment uniformity (over-segmentation). Then the unification of similar adjacent segments occurs until an image partition into uniform regions of maximum size is obtained.
The disadvantages of these methods are the low speed of data processing, the need for increased memory resources, and their use only in the case of a single reference image.
The method of Markov field modeling [G. R. Cross and A. K. Jain, “Markov Random Field Texture Models”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1983; S. German and D. German, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images”, IEEE Transactions on Pattern Analysis and Machine Intelligence, 1984; R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwala, M. Tappen, C. Rother, “A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-based Priors”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 6, June 2008] is based on the assumption that the color of each point of the image depends on the colors of a certain number of adjacent points. A summary of the image model is proposed. It is also possible to make a summary for texture segmentation [Y. Deng, B. S. Manjunath, and H. Shin, “Color Image Segmentation”, CVPR 1999].
A disadvantage of this method is its difficulty of implementation.
Methods based on edge detection operators [M. Jacob, M. Unser, “Design of Steerable Filters for Feature Detection Using Canny-like Criteria”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 8, pp. 1007-1019; Atilla Ozmen and Emir Tufan Akman, “Edge Detection Using Steerable Filters and CNN”, 2002] utilize segmentation which consists in finding region boundaries and is well solved for halftone images. A halftone image is regarded as a function of two variables, and it is presumed that the region boundaries correspond to the maximums of this function's gradient. In order to find them, a differential geometry is used. To increase the resistance against noise, before using filtration an image is usually blurred. Due to the commutativity of Laplace's operator and Gauss's filter, it is possible to perform blurring and boundary finding at the same time.
A disadvantage of this method is a lack of resistance to image noise. Moreover, since the definition of a boundary is different for each task, it is necessary to choose a revision method for filtration results each time that boundary finding methods are applied.
Optimization methods [Y. Deng, B. S. Manjunath, and H, Shin, “Color Image Segmentation”, CVPR 1999] consist of partitioning an image into uniform regions which becomes an optimization task. For this, the segmentation task is formulated as a task of searching for partitioning of an image which has certain qualities, then a function is introduced which reflects the degree of conformity of the obtained segmentation to the presented requirements. For instance, a segmentation quality function is introduced which uses the distribution of colors on an image.
Disadvantages of these methods are their labor intensiveness and high requirements for the resources of mobile devices.
Common disadvantages of all these methods are:                inapplicability of an obtained segmentation to the tasks of stitching due to the fact that methods can work only on one reference image (region growing methods, fragmentation-blending, edge detection)        system resource requirements which are inapplicable for mobile devices—memory and operating speed of a computing unit (clustering methods, region growing, fragmentation-blending, Markov random field, graph theory method, optimization approaches)        poor operation on noisy images (clustering methods, edge detection)        absence of consideration of the spatial arrangement of points (clustering).        
The graph cut minimization method is the closest to the presented solution [U.S. Pat. No. 6,744,923, “System and Method for Fast Approximate Energy Minimization via Graph Cuts”; and R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwala, M. Tappen, C. Rother “A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-based Priors”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 6, June 2008]. An image is presented in the form of a weighted graph with the nodes in the image points. The weight of the graph links reflects the similarity of points (for instance, the distance between the points by a certain metric). Image partitioning is modelled by graph cuts. Graph theory methods introduce the function of the cut “cost” which reflects the quality of obtained segmentation. Thus, the task of image partitioning into uniform regions reduces to the optimization task of finding a minimum-cost graph cut. Apart from color uniformity and segments' texture, this method allows for controlling the form of segments, their size, complexity of boundaries, etc. There are various methods for finding a minimum-cost cut: greedy algorithm (at each stage choosing those links which would make the total cut cost minimal), methods of dynamic programming (guaranteeing the choice at each stage of an optimal link resulting in an optimal path), etc.
A disadvantage of this method is the large number of calculations required in order to obtain a solution which decreases the operating speed as well as the need to provide a large amount of additional memory for finding an optimal cut.